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Abstract — Inspired by some new advances on normal factor 
graphs (NFGs), we introduce NFGs as a simple and intuitive 
diagrammatic approach towards encoding some concepts from 
linear algebra. We illustrate with examples the workings of such 
an approach and settle a conjecture of Peterson on the Pfaffian. 



I. Introduction 

The application of graphical modeling frameworks and of 
the computational tools therein has undoubtedly revolutionized 
the field of coding and information theory. In particular, 
factor-graph (FG) based modeling of codes |1J has facilitated 
efficient message-passing algorithms in decoding capacity- 
achieving codes, and normal reaUzations of codes on graphs 
||2l have brought to surface an elegant duality property of 
codes in a graphical context. It is also remarkable that these 
advances in the information theory community have resonated 
with the recent development in the computer science and 
statistical physics communities, where joint efforts are made 
towards developing efficient algorithmic solvers for various 
intractable computational problems and towards characterizing 
the hardness of such problems. For example. Valiant ||3] devel- 
oped what he calls "holographic algorithms" as polynomial- 
time solvers for a number of problem families which were 
previously unknown to be in P. In a related work H, Chernyak 
and Chertkov characterized a rich family of binary graphical 
models on planar graphs for which the computation of the 
partition function is easy. Valiant's holographic algorithms 
have also recently found applications in the information theory 
community for solving certain constrained-coding capacity 
problems Q. 

Inspired by the holographic algorithms and making a joint 
effort with Forney ||6l, the authors of ||2] have recently re- 
formalized the notion of normal factor graphs (NFGs) by 
introducing the "exterior function" semantics, which unifies 
Forney's duality result in normal realizations ||2] and the so- 
called "holant theorem" used by Valiant U to establish the 
holographic algorithms. This has ignited, most recently, the 
work of Forney and Vontobel 1:8], which pushes further the 
power of NFGs as an elegant graphical model. 

In this paper, we point out yet another direction that NFGs 
may demonstrate their advantages, namely, their use as a linear 
algebraic tool. Much of this work is motivated by the notion 
of "trace diagrams," which has been recently recognized in the 
mathematics community as an elegant and intuitive "diagram- 
matic approach" towards expressing notions in linear algebra 
(HI, QO), Ol, US, ini- Our main objective is to demonstrate 



how NFGs can be used to provide diagrammatic proofs of 
some results, and also to show their ability to generalize 
trace diagrams. Many of the identities used as examples are 
reproductions of results from lfT3l in the language of NFGs 
and we make no claim of the originality of these results. 
However, we believe that the NFG approach is more general 
than trace diagrams and may potentially provide a wider range 
of algebraic tools. Indeed, the lucid approach of NFGs enables 
us to prove a conjecture of Peterson on the Pfaffian ||T4| IfTZl . 

The remainder of this paper is structured as follows. Sec- 
tion [n] is primarily a review of the NFG framework under 
the exterior-function semantics. Section [III] introduces some 
additional notions of NFGs which will be useful in deriving 
the results of this paper The main results and example 
identities are presented in Section |IV] The paper is concluded 
in Section [V] 



II. Preliminaries 



A. Some Notation 



In this subsection, we briefly list some of our notations. 
All functions have finite domains (which might be different) 
and assume their values from the same field F. A univariate 
function is written as f{x), which will also denote the xth 
component of a vector /. That is, / will be used to denote 
a univariate function or a vector, and the distinction will be 
clear from the context - all vectors in this work are column 
vectors. Similarly, a bivariate function is denoted A{xi,X2), 
which will also be used to denote the (.Ti,a;2)th element of 
a matrix A. For any matrix A, we will use A'^ to denote its 
transpose, and if A is a square matrix, we use dot (A) and 
tr{A) to denote its determinant and trace, respectively. 

For any positive integer n, let N = {!,..., n} and let 
Sn be the symmetric group on N. A permutation a G Sn 
such that <t(j) = ij for all j G N will be written as 

^ I . Finally, we use e to denote the 
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Levi-Civita symbol, which is defined as 
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for all {xi,...,Xn) G A^". Note that the domain of e is 
specified by the number of its arguments and hence we need 
not be explicit about it. 



B. Normal Factor Graphs 

Let V, E, and D be disjoint finite sets and {Xi : i e EUD} 
be a collection of finite alphabets. Further, for any 2 C E IJ 
D, we use xi to denote the variables set {xi : i £ 1} where 
Xi € Xi for all i G I. A normal factor graph (NFG) ||2| is a 
quadruple £7 — (V, i?, £>, fv) defined as follows. 

• (V, E) is a graph with vertex set V and edge set E where 
an edge is incident on exactly two vertices. 

• is a set (possibly empty) of dangling edges where in 
contrast to a regular edge, a dangling edge is incident on 
exactly one vertex in V. 

• Using £{v) = {e G EUD : e is incident on v} to denote 
the set of edges and dangling edges incident on the vertex 
V € V, each vertex v G V is associated with a local 
function involving precisely the variables set xgi^^y 
and fv is the collection of all local functions, namely, 
the set {/t, : u e V}. 

Given an NFG Q = (V, E, D, fv), we may associate with 
it the unique form 

xe v£V 

We refer to this as the sum-of-products form represented by Q. 
Clearly, the sum-of-products form is a function of xd, which 
we refer to as the exterior function realized by Q and denote it 
by Zg{xD)- It is not hard to see that while a sum-of-products 
form is represented by a unique NFG, an exterior function 
may be realized by an infinite class of NFGs. Finally, we say 
that two NFGs Qi and Q2 are equivalent, and write Qi — Q2, 
if Zg-^ ~ Zg^. In some subsequent figures we may equate an 
NFG to a function /, which strictly speaking is not correct, 
but what we really mean is that Zg = /. Such practice is 
unlikely to raise confusion and makes some of the analysis 
more transparent. 

A particular sum-of-products form that will be useful in 
developing some of the tools in this paper is what we refer to 
as the "simple" sum-of-products form. We define the simple 
sum-of-products form, denoted (-j-), as the sum-of-products 
form involving exactly two functions with the summation 
being over all common variables. More explicitly, for any 
functions f(xx) and g(xj), the simple sum-of-products form 
is defined as 



f 



m = E /(•^x).g(.T^). 



xxnj 

Fig. [1] shows the NFG Q representing In this figure 

each set of variables is treated as a single variable and hence 
represented by a single edge. If such a set is empty, the 
corresponding edge (or dangling edge) simply disappears from 
the figure. With this understanding, the NFG in Fig. [T] may 
encode several notions from linear algebra, namely: 

• m J' = 0, then Q corresponds to the tensor product. If 
further, 

-1 = and J ^ %, then the tensor product becomes 

the scalar-vector product. 
- X = J = then the tensor product becomes the 

scalar-scalar product. 

• In J, I\J, and J\I are all non-empty, then Q corre- 
sponds to the matrix-matrix product. 



Fig. 1: The NFG Q representing the simple sum-of-products form 
{f\g) where 1\J = {i e J : i ^ J7} and J\I ^ {j £ J : H I}. 



. inj ^ 0, I\J ^ 0, and J\I = 0, then g coiTesponds 
to the matrix-vector product. 

. Z = J 7^ (i.e., inj 7^ and I\J = J\I = 0), then 
Q corresponds to the vector dot product. 

In this work we find it helpful to adopt the notion of 
"ciliation" lfT3l to explicitly indicate the local orderings of 
variables at each node. To this end, we add a dot on each node 
to mark the edge carrying the local function's first argument 
and assume the rest of arguments are encountered in a counter- 
clockwise manner, cf. Fig. |2] We will not insist on ciliations 
in every occasion and use them only to facilitate the analysis. 






(a) (b) (c) 

Fig. 2; Three assignments of cihations illustrating different orderings 
of arguments: (a) f{xi,...,Xn), (b) f{xk+i,.--,x„,xi,...,Xk) 

and (C) f{xi,Xn,Xn-l, ■ ■ ■ ,X2). 

As an example of 
the effect of different 
ciliation arrangements 
on the realized exte- 
rior function, consider 
the NFGs in Fig. [3] 
where A and B are 
n X n matrices. Then, 
it is easy to verify that 
Zg^ ~ AB, Zg^ = 

AB^, Zg, = A^B'^, 

and Zg^ = A^B, when Zg., i = 1,...,4, is viewed as 
a matrix with rows and columns indexed by xi and X2, 
respectively, and the product is the regular matrix product. 
We end this subsection 



Fig. 3: Different arrangements of cilia- 
tions: (a) Qi, (b) 02, (c) Gs, and (d) t/4. 



with an example. Consider 
the NFG g in Fig. g] (a) 
where A is an n x n matrix. 
Then from the definition of 
the exterior function, we 
have 




A 




A 



B 



(a) 



(b) 



tr(A). 



Fig. 4: (a) An NFG that realizes 
tr(A) and (b) an NFG that illus- 
trates \.r{AB) = \.v{BA). 



This is the reason behind the name "trace diagrams" in ||9], 
ifTOl . Ifm . lfT2l . |[T3l . Briefly, a trace diagram is an NFG 
where each node has degree one, degree two, or is associated 
a Levi-Civita symbol. As another example, let ^ be an m x n 
matrix and i? be an n x m matrix. Then traversing the NFG in 
Fig- 111(b) counter-clockwise starting from the upper edge and 
then starting from the lower edge illustrates the well-known 
identity, \i{AB) = tr(BA). 



C. Vertex Grouping and Vertex Splitting 

Given an NFG Q, we may group any two vertex functions of 
Q, say / and g, and replace them with a single function node 
realized by (/|.g). We refer to this process as vertex grouping 
- also known as "closing the box" flTll . Conversely, if is a 
vertex node of Q that is realized by {f\g) for some / and g, 
then we may split the vertex node h into two function nodes 
/ and g. We refer to this as vertex splitting - also known 
as "opening the box" ifTSll . A graphical illustration of vertex 
grouping and vertex splitting is shown in Fig. |5] Note that 
when we put a dashed box around / and g we mean they are 
replaced with the single function node {f\g), in other words. 
Figs. [5] (b) and (c) refer to the same NFG. 



/ 



9 



Grouping 



Splitting 



/ 



g 



(/Iff) 



(a) (b) (c) 

Fig. 5: Vertex grouping and vertex splitting 

Since the range of all the functions in an NFG is a field, 
the following lemma, which will be the main proving tool to 
most of the subsequent results, is cleaiQ. 

Lemma 1. The exterior function realized by any NFG is 
invariant under vertex grouping and vertex splitting. 

Clearly one may group an arbitrary number of nodes in an 
NFG by recursively grouping pairs of vertices; ultimately, one 
may replace an NFG Q with a single function node realizing 
the exterior function Zg. 

III. Derived NFGs: Scaling, Addition, and 
Subtraction of NFGs 

We have already seen vertex grouping and splitting as a 
means of manipulating NFGs. Now, given two NFGs Qi and 
Q2 with disjoint dangling edge sets, one of the most basic 
ways to obtain a new NFG, say Q, from Qi and Q2 is to 
stack Qi and Q2 beside each other Grouping the vertices of 
Qi and Q2, it is clear that Zg is the tensor product of Zg-^ 
and Zg^ . Of particular interest is the case when Q2 is a single 
node of degree zero. In this case Zg^ is a constant, say A, 
and so Zg = XZg-^. Moreover, we write Q = XQi and say 

is a scaled version of Qi with A being the scaling factor 
Graphically, we show scaling by putting the scaling factor 
beside the NFG0 

Given two NFGs Qi and Q2 with the same set of dan- 
gling edges, we may graphically construct a compound NFG, 
denoted Qi + G2, by drawing the two graphs with the plus 
sign, "+", between them. Fig. |6l We use this compound NFG 
to represent the function Zg^ + Zg^, and more formally, 
we say the compound NFG realizes Zg-^ + Zg^. Finally, the 
"subtraction of Q2 from Qi" compound NFG, denoted Q1—Q2, 
is defined as Qi + {—1)Q2- 

' We remark that all is needed for Lemma [T]to hold is the distributive law, 
and so it is still true even when the range of the local functions of the NFG 
is a semi-ring rather than a field. 

-An equivalent way to obtain Q directly from Q\ is to replace any function 
node, say /, in Qi with the function node A/ in Q, and to leave everything 
else the same. 




3^1 I 1 X2 



Fig. 6: Addition of two NFGs. 

IV. Some Linear Algebra Using NFGs 

The main purpose of this section is to demonstrate how 
NFGs can be used to establish various identities from lin- 
ear algebra in a simple and intuitive diagrammatic manner 
Throughout the section, we will liberally make use of the 
following lemma about the Levi-Civita symbol, which easily 
follows from its definition. 

Lemma 2. For any positive integer n, it holds that 

e{xi,X2, ■ ■ . ,a;„_i,x„) = (-l)""^e(x2, 2:3, . . . ,a;„,xi), 

and so, when n is odd, e becomes invariant under cyclic-shifts 
of its arguments. 

A. Kronecker Delta 



The Kronecker delta, denoted S{x), is 
the indicator function evaluating to one 
if and only if a; = 0. An NFG realizing 
S{x ~ y) is shown in Fig. [T] (a), which 
from now on will be abbreviated as the 
NFG in (b). 



y y 
(a) (b) 
Fig. 7: Both NFGs 
in (a) and (b) real- 
ize 5{x — y). 



Fig. 8: Cross prod- 
uct. 



B. Cross Product 

In this subsection we look at the cross 
product of vectors of length 3. Let Q be as 
in Fig. |8] then direct computation shows 
that 

Zg(l) =u(2)w(3) -it(3)t;(2), 
Zg{2)=u{i)v{l)~u{l)v{i), 
Zg(3) =u(l)w(2) -it(2)w(l). 

I.e., when viewed as vectors in F'^, the 
exterior function Zg is the cross product 

u y. V. 

A well-known fact, and straightforward to prove, is the 
following identity about the contraction of two Levi-Civita 
symbols, namely 

^'^{yi,y2,t)£{t,x2,xi) = 
t 

5{xi - y2)5{x2 - yi) - S{xi - yi)S{x2 - 2/2)- 

A graphical illustration of the Levi-Civia contraction identity 
is shown in Fig. |9] 

Gluing single variate nodes w, s, u, and v, respectively, 
on the dangling edges xi, X2, yi, and y2 in Fig. |9] results in 
Fig. [To] From the previous analysis (in particular. Lemmas [T] 
and[2]i, the following identities are then clear: 

(u X v) ■ {s X w) ~ {{u X v) X s) ■ w — {w X [u X v)) ■ s 
~ ((s X w) X u) ■ V = {v X (s X w)) ■ u 
= {u ■ s){v ■ w) — {u ■ w){v ■ s). 



yi 2/2 

Fig. 9: Contraction of two Levi-Civita symbols. 








C\ \D\ = ^{AD^ BC^ ) - tr(BC^ )tx{AD 





Fig. 10: A proof of {uxv)-[sxw) = - ■ ■ = {u-s){v-'w) — {u-w){v-a). 

Many identities regarding the cross product may be obtained 
in the same way. For instance, let rria, mi,, nic, and md be 
arbitrary positive integers and let {ui : 1 < i < rua}, {bi : 1 < 
i < rrib}, {ci : 1 < i < md, and {di : 1 < i < nid} be four 
collections of length-three vectors. Further, let A be the matrix 
whose ith column is a.i, i.e., A = (a^ : 1 < i < nia), and 
similarly let B ^ {hi : 1 < i < mi,), C = {ci : 1 < i < mc), 
and D = {di : I < i < md). We claim that if ma = md = m 
and mb — mc — m' for some m and m' , then 



^ (a, xbj)- (cj X ) = tr( AD^BC^) -tr(BC^)tr( AD^ 

i=l 3 = 1 



A detailed proof of this identity is shown in Fig. [TT] 

We emphasize that in practice, one does not need to go 
through all the steps in Fig. (TT] i.e., one should be able to 
deduce the identity easily from the last two NFGs. Slight dct(yl_B) 
variations of the NFG before the last equivalence in Fig. [TT] 
may be used to prove more identities about the cross product. 
For instance, when ma = mi, = m and mc = md = m' for 
some m and m' , Fig. [12] (a) shows that 




C. Determinant 

For any n x n matrix A, the determinant is defined as 

det{A)= ^ sgn(a)n^O-,^(j))- 

From the definitions of exterior function and determinant, it 
follows that 



dct(A) 



From this, several properties of the determinant may be 
obtained, for instance 







det{A)_/^\ = dct(yl)det(B) 




^ ^(a, X hi) ■ {cj X dj) = tr{AB'^DC^)~tT{AB'^CD'^), 



shows that dct{AB) = det{A) dct{B) for any nxn matrices 
A and B. Another example follows from 



t=l 3 = 1 

and when ma ~ 1 and mi, - 
proves X]^i(ai x b^) x Ci 

We invite the reader to verify some of the identities above 
using traditional methods and contrast with the current dia- 
grammatic approach. 



mc = m for some rn, Fig.[T2](b) 

= {BC^) ai - tr {BC^ 



ai. 



det{A 




dct{A) 




showing det(^"^) = det{A). As a final example, we have 




N 




[e] 










which proves 

(ai X 02) • 03 ^ (02 X as) • fli = (as x ai) • 02 = dct(^), 

where A is the 3x3 matrix (ai 02 03) and for each i G 
{1, 2, 3}, S,ix) = Six - i) for all x G {1, 2, 3}. 



D. Pfaffian 

Given a 2n x 2n skew-symmetric matrix A, the Pfaffian of 
j4, denoted Pf(A), is defined as 

Pf(A) = — ^ ^ sgn(a)n^H2*-l),^(2*)). 



(TGS2. 



Before we proceed, we need the following lemma. 

Lemma 3. For any positive integer n, let r G S2n be such 
that T{2k - 1) = fc and T{2k) = 2n - (fc - 1), 1 < fc < n. 
Then r is an even permutation. 

Proof: For brevity, we use the tuple (ii,...,i2„) to 

refer to the permutation ^ f ^ ' given the 

identity permutation (1, . . . , n, 71 + 1, . . . , 2n), it is possible to 
reflect the second half of the permutation so that it becomes 
(1, . . . , 71, 2n, . . . , 7i + 1); this clearly can be done using [^J 
swaps. Next, we interleave the two halves of the 27T,-tuple by 
performing the following ti — 1 steps: Starting with fc = 1 and 
ending with fc = ti — 1, at step fc imagine a window that starts 
just before the 2fcth element of the tuple and ends just after 
the 71 + fcth element (i.e., it covers n — k + 1 elements). For the 
elements of the tuple covered by the window, we perform a 
cyclic-shift by one position to the right. It is easy to check that 
after the (n — l)th step we arrive to our desired permutation. 
Clearly, a one-position cyclic shift on m elements can be 
accomplished using m—1 swaps. Hence, interleaving the tuple 
requires X]m=2(™ ^ 1) — "^"2'^'' swaps. Therefore, in total 
our permutation can be written in terms of J + "^"^'^'^ swaps 
and the claim follows by noting that this number is even for 
any positive integer 77. ■ 
The following proposition affirmatively proves Peterson's 
conjecture on the Pfaffian lfT4l lfT2l . 

Proposition 1. Let A be a 2n x 2n skew-symmetric matrix 
and let Q be as in Fig. [73 Then = 77! • 2" • Pf(A). 




Proof: We have 

n 

Zg = ^ e(ii,---,««,jn,---,ji) J|^(ifc,jfc) 

(ji,....i„j„....,ii)ew2- fc=i 

n 

= E ^E<^)X{A{a{k),a{2n-{k-l))). 



0-652 



fc=l 



Now let (T = 



n 77 + 1 

Jn 

277-1 277 



2n 
ii 



and 



and note that 



Fig. 13: The NFG Q from Proposition [T] 



1 2 

12 3 4 

h jl 12 j2 

a' = a o T where t is as in Lemma [3] Hence, sgn{a') 
sgn(o') •sgn(T) = sgn((T), where the last equality follows from 
Lemma |3] Further, note that for all fc G {1, . . . , 77} we have 
cr(fc) =ik^ cr'(2k - 1) and a{2n - (fc - 1)) = jk = cr'(2fc-). 
Finally, it is clear that as a runs over S2n, o"' runs over S2n- 
Hence, Zg = Ea'es,,. sgni<j')]XU A{a' {2k " l),<^'(2fc)), 
and the claim follows by the definition of the Pfaffian. ■ 

V. Conclusion 

This paper presents NFGs as an intuitive tool that facilitates 
proofs and enhances understanding of some notions and iden- 
tities in linear algebra. We feel that this new look at NFGs, 
together with the related work |]8], may result in a better un- 
derstanding of NFGs as a model of codes and provide deeper 
understanding of the message-passing algorithms. Further, it is 
our hope that the use of NFGs as an algebraic tool may open 
doors for developing new algorithmic tools that are useful in 
a broader range of disciplines. 
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